homework 8, version 1

Submission by: Jazzy Doe (jazz@mit.edu)

Bonus homework 8: Raytracing in 3D

18.S191, fall 2020

This week we will recreate the 3D raytracer from the lectures, building on what we wrote for the 2D case. Compared to the past weeks, you will notice that this homework is a little different! We have included fewer intermediate checks, it is up to you to decide how you:

• split up the problem is smaller functions

• check and visualize your progress

To guide you through the homework, you can follow along with this week's live coding session. Feel free to stay close to James's code, or to come up with your own solution.

With the raytracer done, we will be able to recreate an artwork by M.C. Escher, scroll down to have a sneak peek!

For MIT students: this homework is optional.

Feel free to ask questions!

Let's create a package environment:

From the last homework

Below we have included some important functions from the last homework (Raytracing in 2D), which we will be able to re-use for the 3D case.

There are some small changes:

1. The concept of a Photon now carries color information.

2. The Sphere is no longer a pure lens, it contains a Surface property which describes the mixture between transmission, reflection and a pure color. More on this later!

3. The refract function is updated to handle two edge cases, but its behaviour is generally unchanged.

Outside of these changes, all functions from the previous homework can be taken "as-is" when converting to 3D, cool!

Intersections:

Reflect and refract:

Surface (new)

Camera and Skyboxes

Now we can begin looking into the 3D nature of raytracing to create visualizations similar to those in lecture. The first step is setting up the camera and another stuct known as a sky box to collect all the rays of light.

Luckily, the transition from 2D to 3D for raytracing is relatively straightforward and we can use all of the functions and concepts we have built in 2D moving forward.

Firstly, the camera:

Camera

For the purposes of this homework, we will constrain ourselves to a camera pointing exclusively downward. This is simply because camera positioning can be a bit tricky and there is no reason to make the homework more complicated than it needs to be!

So, what is the purpose of the camera?

Well, in reality, a camera is a device that collects the color information from all the rays of light that are refracting and reflecting off of various objects in some sort of scene. Because there are a nearly infinite number of rays bouncing around the scene at any time, we will actually constrain ourselves only to rays that are entering our camera. In poarticular, we will create a 2D screen just in front of the camera and send a ray from the camera to each pixel in the screen, as shown in the following image:

Because we are not considering camera motion for this exercise, we will assume that the image plane is constrained to the horizontal plane, but that the camera, itself, can be some distance behind it. This distance from the image plane to the camera is called the focal length and is used to determine the field of view.

From here, it's clear we need to construct:

1. A camera struct

2. A function to initialize all the rays being generated by the camera

Let's start with the struct

Now we need to construct some method to create each individual ray extending from the camera to a pixel in the image plane.

Nothing yet... time to add some objects!

Skybox

Now that we have the concept of a camera, we can technically do a fully 3D raytracing example; however, we want to ensure that each pixel will actually hit something – preferrably something with some color gradient so we can make sure our simulation is working!

For this, we will introduce the concept of a sky box, which is standard for most gaming applications. Here, the idea is that our entire scene is held within some additional object, just like the mirrors we used in the 2D example. The only difference here is that we will be using some texture instead of a reflective surface. In addition, even though we are calling it a box, we'll actually be treating it as a sphere.

Because we have already worked out how to make sure we have hit the interior of a spherical lens, we will be using a similar function here. For this part of the exercise, we will need to construct 2 things:

1. A skybox struct

2. A function that returns some color gradient to be called whenever a ray of light interacts with a sky box

So let's start with the sky box struct

Now we have the ability to create a skybox, the only thing left is to create some sort of texture function so that when the ray of light hits the sky box, we can return some form of color information. So for this, we will basically create a function that returns back a smooth gradient in different directions depending on the position of the ray when it hits the skybox.

For the color information, we will be assigning a color to each cardinal axis. That is to say that there will be a red gradient along $x$, a blue gradient along $y$, and a green gradient along $z$. For this, we will need to define some extent over which the gradient will be active in 'real' units. From there, we can say that the gradient is

$$\frac{r+D}{2D},$$

where $r$ is the ray's position when it hits the skybox, and $D$ is the extent over which the gradient is active.

So let's get to it and write the function!

Let's set up a basic scene and trace an image! Since our skybox is spherical we can use the same intersect method as we use for Spheres. Have a look at the intersect method, we already added SkyBox as a possible type.

To create this image, we used the ray tracing function bewlow, which takes in a camera and a set of objects / scene, and...

1. Initilializes all the rays

2. Propagates the rays forward

3. Converts everything into an image

Writing a ray tracer

It's your turn! Below is the code needed to trace just the sky box, but we still need to add the ability to trace spheres.

We recommend that you start by just implementing reflection - make every sphere reflect, regardless of its surface. Make sure that this is working well - can you see the reflection of one sphere in another sphere? Does our program get stuck in a loop?

Once you have reflections working, you can add refraction and colored spheres. In the 2D example, we dealt specifically with spheres that could either 100% reflect or refract. In reality, it is possible for objects to either reflect or refract, something in-between. That is to say, a ray of light can split when hitting an object surface, creating at least 2 more rays of light that will both return separate color values. The color that we perceive for that ray is the combination both of these colors - they are mixed.

A third possibility explored in the lecture is that the objects can also have a color associated with them and just return the color value instead of reflecting or refracting.

You can choose! After implementing reflection, you can implement three different spheres (you can modify the existing code, create new types, add functions, and so on), a purely reflective, purely refractive or opaquely colored sphere. You can also go straight for the more photorealistic option, which is that every sphere is a combination of these three - this is what we did in the lecture.

Below, we create a scene with a number of balls inside of it. While working on your code, work in small increments, and do frequent checks to see if your code is working. Feel free to modify this test scene, or to create a simpler one.

Bonus: Escher

If you managed to get through the exercises, we have a fun bonus exercise! The goal is to recreate this self-portrait by M.C. Escher:

It looks like M.C. Escher is a skillful raytracer, but so are we! To recreate this image, we can simplify it by having just two objects in our scene:

• A purely reflective sphere

• A skybox, containing an image of us!

Let's start with our old skybox, and set up our scene:

👆 You can modify escher_cam to increase or descrease the resolution!

Awesome! Next, we want to set an image as our skybox, instead of a gradient. To do so, we have written a function that converts the x,y,z coordinates of the intersection point with a skybox into a latitude/longitude pair, which we can use (after scaling, rounding & clamping) as pixel coordinates to index an image!

Great! It's like the Earth, but distorted. Notice that the continents are mirrored, and that you can see both poles at the same time.

Okay, self portrait time! Let's take a picture using your webcam, and we will use it as the skybox texture:

👀 wow! It's Planet Jazzy Doe, surrounded by even more Jazzy Doe.

When you look at the drawing by Escher, you see that he only occupies a small section of the 'skybox'. Behind Escher, you can see his cozy house, and in front of him (i.e. behind the glass sphere, from his persective), you see a gray background.

What we need is a 360° panoramic image of our room. One option is that you make one! There are great tutorials online, and maybe you can use an app to do this with your phone.

Another option is that we approximate the panaroma by padding the image of your face. You can pad the image with a solid color, you can use a gradient, you can use the earth satellite image, you can add noise, and so on. Here is an example of what I made.

Let's put it all together!

Function library

Just some helper functions used in the notebook.